Visual sensor abnormality cause estimation system

ABSTRACT

A camera abnormality cause estimation system for estimating the causes of abnormalities in a camera in a production system in which the camera controls a robot. The production system includes a robot, a camera that detects visual information of the robot or the surrounding thereof, and a controller that controls the robot based on an image signal obtained by the camera. The camera abnormality cause estimation system estimates the causes of abnormalities in the camera and includes an environment information acquisition unit that acquires environment information of the camera, and an abnormality cause estimation unit that estimates a probability that each of a plurality of predetermined abnormality cause items is the cause of an abnormality in the camera for the respective abnormality cause items using the environment information acquired by the environment information acquisition means and displays the estimated probability on a display unit for the respective abnormality cause items.

This application is based on and claims the benefit of priority fromJapanese Patent Application No. 2017-003343, filed on 12 Jan. 2017, thecontent of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the invention

The present invention relates to a visual sensor abnormality causeestimation system. More specifically, the present invention relates to avisual sensor abnormality cause estimation system for estimating thecause of abnormalities in visual sensors installed in a productionsystem.

Related Art

Visual sensors are installed in various production devices such as arobot that conveys and processes a target object or an inspection devicethat determines the quality of a target object in order to obtain visualinformation of the target object. For example, a conveying robot thatconveys a target object from, a predetermined position to apredetermined position detects the position, the attitude, and the likeof the target object by performing image processing such as templatematching on visual information obtained by visual sensors, grasps thetarget object at an appropriate position, and conveys the target objectto a predetermined position. In a production system in which visualsensors are installed in this manner, when a desired image is notobtained by a visual sensor due to certain abnormalities in the visualsensor, a production device controlled using visual information obtainedby the visual sensor may be unable to perform its functionappropriately.

For example, Patent Document 1 discloses a device that detectsabnormalities in a visual sensor. More specifically, Patent Document 1discloses a device that determines whether a visual sensor operates insynchronization with an external synchronization signal to detectabnormalities in the visual sensor in a system which outputs an externalsynchronization signal to the visual sensor and processes image dataoutput from the visual sensor in synchronization with the externalsynchronization signal. According to the device disclosed in PatentDocument 1, it is possible to specify whether the cause of abnormalitiesin the visual sensor is attributable to disconnection of a signal linethat delivers an external synchronization signal to the visual sensor orthe visual sensor itself breaks down and is unable to operate insynchronization with the external synchronization signal.

For example, Patent Document 2 discloses a device that detects anabnormal image from images obtained by a visual sensor. Morespecifically, Patent Document 2 discloses a device that detects anabnormal image obtained by a visual sensor by comparing an imagestatistical value calculated using images captured by the present visualsensor with an image statistical value calculated using images capturedwhen the visual sensor operates normally.

Patent Document 1: Japanese Unexamined Patent Application, PublicationNo. 2013-46350

Patent Document 2: Japanese Unexamined Patent Application, PublicationNo. 2006-352644

SUMMARY OF THE INVENTION

However, when an abnormality occurs actually in a visual sensorinstalled in a production system, an operator needs to specify aspecific cause of the abnormality in the sensor in order to allow theproduction device to return to an appropriate state. However, there area wide range of causes of abnormalities in the visual sensor. Morespecifically, examples of the causes include (A) a case in which thevisual sensor itself breaks down and is unable to perform its function,(B) a case in which a trouble occurs in a cable that connects the visualsensor to other devices, and (C) a case in which the cause results froma change in ambient environment of the visual sensor. When the cause ofabnormalities in the visual sensor is (A) or (B), an operator canspecify the cause of the abnormality using such a device as disclosed inPatent Document 1, for example.

However, when the cause of abnormalities in the visual sensor is (C),that is, when the cause results from, a change in ambient environment ofthe visual sensor, it is difficult to specify a specific cause ofabnormalities clue to the following reasons. For example, since anabnormality in the visual sensor resulting from such a case may occureven when the visual sensor has not broken down, it is difficult tospecify the cause by monitoring the state of the visual sensor.Moreover, since an abnormality in the visual sensor resulting from sucha case may occur under a specific condition, it is difficult toreproduce an environment which caused an abnormality. Moreover, there isa case in which an abnormality occurs in a visual sensor after theelapse of a short period after a specific event which caused anabnormality in the visual sensor occurs. In this case, it is difficultto specify a causal dependency between a specific event and anabnormality in the visual sensor. Due to this, conventionally, when anabnormality occurs in a visual sensor, an operator has to check apresumed cause by a round robin method and it may take a considerableamount of time for the production device to return to a normal state.

An object of the present invention is to provide a visual sensorabnormality cause estimation system for estimating the cause ofabnormalities in a visual sensor in a production system in which thevisual sensor is installed to control a production device.

(1) A production system (for example, a production system S to bedescribed later) includes a production device (for example, a robot 5 tobe described later), a visual sensor (for example, a camera 6 to bedescribed later) that detects visual information on the productiondevice or a surrounding thereof, and a controller (for example, acontroller 7 to be described later) that controls the production deviceon the basis of the visual information obtained by the visual sensor. Avisual sensor abnormality cause estimation system (for example, anabnormality cause estimation system 1 or 1A to be described later) ofthe present invention estimates the causes of abnormalities in thevisual sensor and includes: an environment information acquisition means(for example, an environment information acquisition means 2 to bedescribed later) that acquires environment information of the visualsensor; and an abnormality cause estimation means (for example, anarithmetic unit 32 or 32A or an abnormality cause estimation unit 37 or37A to be described later) that estimates a strength of correlationbetween an abnormality in the visual sensor and each of a plurality ofpredetermined abnormality cause items using the environment informationacquired by the environment information acquisition means and displaysthe estimated strength of correlation on a display means (for example, adisplay unit 33 to be described later) for the respective abnormalitycause items.

The expression “abnormality in the visual sensor” in the presentinvention is defined as the inability of the visual sensor to obtaindesirable visual information. Moreover, the expression “desirable visualinformation” in the present invention is defined as information withwhich the production device can realize an appropriate function (aconveying operation of grasping a target object at an appropriateposition and an appropriate attitude and conveying the target object toan appropriate position, a function of determining the quality of thetarget object accurately) when the production device is controlled onthe basis of this information. Therefore, in the present invention, itis possible to cope with abnormalities in the visual sensor even whendesirable visual information is not obtained due to a failure in thevisual sensor itself and a trouble in a cable connecting the visualsensor and other devices and when desirable visual information is notobtained due to strong light entering the lens of the visual sensor orvibration of the visual sensor.

(2) In the visual sensor abnormality cause estimation system accordingto (1), the abnormality cause estimation means may estimate the strengthof correlation using the environment information in a period between anabnormality occurrence time point of the camera and a predeterminedprevious time point.

(3) The visual sensor abnormality cause estimation system according to(2) may further include an abnormality occurrence time point specifyingmeans (for example, an image processing unit 71 to be described later)that specify the abnormality occurrence time point on the basis of thevisual information.

(4) The visual sensor abnormality cause estimation system according to(2) may further include: an image processing means (for example, animage processing unit 71 to be described later) that performs apredetermined image processing operation on the visual information; andan abnormality occurrence time point specifying means (for example, arobot operation determination unit 36 or 36A or a camera abnormalitydetection unit 38A to be described later) that specifies the abnormalityoccurrence time point using image processing information obtainedthrough the image processing operation and operation result informationon a result of operations performed by the production device on thebasis of the visual information.

(5) In the visual sensor abnormality cause estimation system accordingto any one of (1) to (4), the environment information acquisition meansmay acquire subdivides the environment of the visual sensor into aplurality of environmental items and acquires environment informationbelonging to each of the environmental items, and each of theabnormality cause items is the same as one of the plurality ofenvironmental items.

(6) In the visual sensor abnormality cause estimation system accordingto (5), the environmental item may include vibration of a specificdevice installed around the visual sensor, a temperature of the specificdevice, and a brightness of the surrounding of the visual sensor.

(7) In the visual sensor abnormality cause estimation system accordingto any one of (1) to (6), the abnormality cause estimation means mayestimate the strength of correlation between an abnormality in thevisual sensor and each of the plurality of abnormality cause items usingthe environment information and information on the change over timethereof as an input.

In the production system, the visual sensor detects the visualinformation of the production device or the surrounding thereof and thecontroller controls the production device on the basis of the visualinformation obtained by the visual sensor. The environment informationacquisition means acquires environment information of the visual sensorincluded in the production device, and the abnormality cause estimationmeans estimates the strength of the correlation between the abnormalityin the visual sensor and each of the plurality of predeterminedabnormality cause items using the environment information acquired bythe environment information acquisition means and displays the strengthof correlation on the display means for the respective abnormality causeitems. In this way, when an abnormality occurs in the visual sensorduring operations of the production system, for example, the operatorcan check the causes of the abnormality from a plurality of abnormalitycause items in descending order of the strengths of correlationdisplayed on the display means using the abnormality cause estimationsystem of the present invention. Therefore, the operator can specify thecause of the abnormality in the visual sensor more quickly than theconventional method in which a presumed cause is checked by around-robin method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram, illustrating a configuration of a production systemin which a visual sensor abnormality cause estimation system accordingto a first embodiment of the present invention is included.

FIG. 2 is a diagram illustrating a specific procedure of operationsrelated to estimation of the cause of abnormalities in a camera.

FIG. 3 is a diagram illustrating an example of an estimation result ofan abnormality cause estimation unit, displayed on a display unit.

FIG. 4 is a diagram illustrating a specific procedure of operationsrelated to estimation of the cause of abnormalities in a camera in anabnormality cause estimation system according to a second embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

Hereinafter, a first embodiment of the present invention will bedescribed with reference to the drawings. FIG. 1 is a diagramillustrating a configuration of a production system S in which a visualsensor abnormality cause estimation system 1 according to the presentembodiment is included.

The production system S includes a robot 5 as a production device, acamera 6 as a visual sensor that detects visual information of the robot5 or the surrounding thereof, a controller 7 that controls the robot 5on the basis of an output of the camera 6, and an abnormality causeestimation system 1 that estimates the cause of abnormalities in thecamera 6.

For example, the robot 5 is a conveying robot that executes a series ofconveying operations of grasping a workpiece W which is one of the partsof a product at a predetermined position according to a control signaltransmitted from the controller 7 and conveying the grasped workpiece Wup to a predetermined position.

The camera 6 is installed on a frame provided in the robot 5 or thesurrounding thereof. The camera 6 images the robot 5 or the workpiece Waccording to a request from the controller 7 to obtain an image signaland transmits the image signal to the controller 7 and the abnormalitycause estimation system 1 at a predetermined interval.

The controller 7 includes an image processing unit 71 and a robotcontrol unit 72. The image processing unit 71 performs a plurality ofimage processing operations on the image signal transmitted from thecamera 6. The image processing unit 71 transmits image processinginformation obtained through the plurality of image processingoperations to the robot control unit 72 and the abnormality causeestimation system 1 at a predetermined interval. Here, examples of theimage processing information transmitted from the image processing unit71 include an image including the workpiece W, the contrast of anoutline of the workpiece W captured in the image, the brightness of thisimage, the score indicating the probability of the workpiece W detectedfrom the image, a detection position of the workpiece W detected fromthe image and a detection error thereof, the presence of a detectionerror of the workpiece W (that is, a position different from the actualposition being detected as the position of the workpiece W in theimage), a black level of the camera 6, and a data error rate of an imagesignal transmitted from the camera 6.

The robot control unit 72 generates a control signal for causing therobot 5 to execute the above-described conveying operation on the basisof the image processing information obtained through the plurality ofimage processing operations of the image processing unit 71 andtransmits the control signal to the robot 5. Moreover, the robot controlunit 72 acquires information on the result of the conveying operationexecuted on the basis of the image signal of the camera 6 as describedabove, on the basis of a signal transmitted from a force sensor or thelike (not illustrated) installed in the robot 5 and transmits theinformation to the abnormality cause estimation system 1 as operationresult information at a predetermined interval. Here, examples of theoperation result information transmitted from the robot control unit 72includes information on whether the robot 5 is able to grasp theworkpiece W in a series of conveying operations and a grasping shiftamount from a predetermined reference position when the robot 5 graspsthe workpiece W.

The abnormality cause estimation system 1 includes an environmentinformation acquisition means 2 that acquires information on theenvironment of the camera 6 and an information processing device 3 thatperforms operations related to estimation of abnormalities in the camera6 using the information transmitted from the environment informationacquisition means 2, the camera 6, the image processing unit 71, and therobot control unit 72 and the cause thereof.

The environment information acquisition means 2 acquires a plurality oftypes of environment information on the environment that may affect thefunctions of the camera 6 at predetermined intervals and transmits theacquired environment information to the information processing device 3.Here, the environment information acquired by the environmentinformation acquisition means 2 is subdivided into thirteenenvironmental items, for example, as illustrated below. The environmentinformation belong to respective items is as follows.

Information on vibration of the camera 6 is classified to Item 1. Theenvironment information acquisition means 2 acquires environmentinformation belonging to Item 1 using an output of a vibration sensorprovided in the camera 6, for example. Information on temperature of thecamera 6 is classified to Item 2. The environment informationacquisition means 2 acquires environment, information belonging to Item2 using an output of a temperature sensor provided in the camera 6, forexample. Information on temperature of a peripheral device providedaround the camera 6 is classified to Item 3. The environment informationacquisition means 2 acquires environment information belonging to Item 3using an output of a temperature sensor provided in a peripheral device,for example. Information on the brightness of the surrounding of thecamera 6 is classified to Item 4. The environment informationacquisition means 2 acquires environment information belonging to Item 4using an output of an illuminance sensor provided in or near the camera6, for example. Information on a date and a time point is classified toItem 5. The environment information acquisition means 2 acquiresenvironment information belonging to Item 5 using a clock, for example.

Information on weather is classified to Item 6. The environmentinformation acquisition means 2 acquires environment informationbelonging to Item 6 using information distributed from a weatherinformation distribution server, for example. Information on an ON/OFFstate of a ceiling light in a room where the camera 6 is installed isclassified to Item 7. The environment information acquisition means 2acquires environment information belonging to Item 7 using an output ofa controller of the ceiling light, for example. Information on settingtemperature of an air-conditioner in a room where the camera 6 isinstalled is classified to Item 8. The environment informationacquisition means 2 acquires environment information belonging to Itemusing an output of a controller of the ceiling light, for example.Information on the type of the workpiece W which is an operation targetof the robot 5 is classified to Item 9. The environment informationacquisition means 2 acquires environment information belonging to item 9using a signal transmitted from the controller 7, for example.Information on an operating state of a peripheral device provided aroundthe camera 6 (more specifically, whether the peripheral device is in anoperating state, a stopping state, or a standby state) is classified toItem 10. The environment information acquisition means 2 acquiresenvironment information belonging to Item 10 using a signal transmittedfrom the peripheral device.

Information on a signal transmitted and received to synchronizeoperations between a plurality of devices installed in the productionsystem S is classified to Item 11. The environment informationacquisition means 2 acquires environment information belonging to Item11 using a signal transmitted from respective devices, for example.Information on a supply voltage of the camera 6 is classified to Item12. The environment information acquisition means 2 acquires environmentinformation belonging to Item 12 using an output of a voltage sensor,for example. Information on alarms output from respective devices isclassified to Item 13. The environment information acquisition means 2acquires environment information belonging to Item 13 using alarmsignals transmitted from respective devices, for example.

The environment information acquired by the environment informationacquisition means 2 includes non-numerical information such as weatherinformation of Item 6 and information on the ON/OFF state of the ceilinglight of Item 7, for example. The environment information acquisitionmeans 2 preferably numericalize such non-numerical information accordingto predetermined rules and transmit the same to the informationprocessing device 3. Specifically, as for the weather information, forexample, “0” may be allocated to “sunny” and “1” may be allocated to“cloudy.” Moreover, as for the information on the ON/OFF state of theceiling light, “1” may be allocated to “ON” and “0” may be allocated to“OFF.”

A change in the environment information acquired using the environmentinformation acquisition means 2 is gentler than a change in the imagesignal transmitted from the camera 6 or the image processinginformation, the operation result information, and the like transmittedfrom the controller 7. Therefore, the environment informationacquisition means 2 preferably acquire the environment information at alonger interval than the interval of the controller 7 generating theimage processing information or the operation result information andtransmit the environment information to the information processingdevice 3.

The information processing device 3 is configured as a computerincluding a storage device 31 that stores various pieces of data,programs, and the like, an arithmetic unit 32 that executes operationsrelated to estimation of abnormalities in the camera 6 and the causethereof using the data stored in the storage device 31, and a displayunit 33 that displays an operation result obtained by the arithmeticunit 32 in a form that can be visually perceived by an operator.

Pieces of time-series data of the image signal transmitted from thecamera 6, the image processing information transmitted from the imageprocessing unit 71, the operation result information transmitted fromthe robot control unit 72, and the environment information transmittedfrom the environment information acquisition means 2 are stored in thestorage device 31. Moreover, information on change over time in theimage processing information and the operation result information, andthe environment information is also stored in the storage device 31.Here, the change over time is a difference value between a value at eachtime point of target information and a value at a time point before apredetermined period.

FIG. 2 is a functional block diagram illustrating a functionalconfiguration of the arithmetic unit 32. The arithmetic unit 32 includesa robot operation determination unit 36 and an abnormality causeestimation unit 37 and estimates the cause of abnormalities in thecamera 6 using these units. Hereinafter, the functions of the respectivemodules 36 and 37 configured to be realized by operations of thearithmetic unit 32 will be described sequentially.

The robot operation determination unit 36 determines whether the robot 5can realize an appropriate function using the operation resultinformation among the pieces of data stored in the storage device 31.More specifically, the robot operation determination unit 36 determineswhether a conveying operation executed by the robot 5 is appropriate onthe basis of the image signal transmitted from the camera 6. Asdescribed above, the operation result information includes informationon success or failure in a series of conveying operations executed bythe robot 5 on the basis of the image signal of the camera 6. Morespecifically, the operation result information includes information onwhether the robot 5 is able to grasp the workpiece W and information ona grasping shift amount. Therefore, the robot operation determinationunit 36 determines that the conveying operation of the robot isappropriate when the robot 5 is able to grasp the workpiece W and thegrasping shift amount is equal to or larger than a predeterminedreference value. Moreover, the robot operation determination unit 36determines that the conveying operation of the robot is not appropriatewhen the robot 5 is unable to grasp the workpiece W or the graspingshift amount is larger than the reference value.

When it is determined that the conveying operation of the robot is notappropriate, the robot operation determination unit 36 specifies theimage processing information used as the input to the robot control unit72 at the time of executing the inappropriate conveying operation fromthe past image processing information stored in the storage device 31and transmits the acquisition time point of the image signal used forthe image processing unit 71 to generate the image processinginformation to the abnormality cause estimation unit 37 as anabnormality occurrence time point of the camera 6.

Although FIG. 2 illustrates a case in which the abnormality occurrencetime point of the camera 6 is specified from the result of the conveyingoperation of the robot by the robot operation determination unit 36, themeans for specifying the abnormality occurrence time point of the camera6 is not limited to this. The abnormality occurrence time point of thecamera 6 may be manually set by an operator and be input to theabnormality cause estimation unit 37. When a certain abnormality occursin the camera 6 and the image signal transmitted from the camera 6 isabnormal, the occurrence of an abnormality in the camera 6 and theabnormality occurrence time point may be specified by the imageprocessing unit 71 performing an image processing operation on the imagesignal from the camera 6. Therefore, the abnormality occurrence timepoint of the camera 6 may be specified by the image processing unit 71on the basis of the image signal transmitted from the camera 6.

In this way, the abnormality cause estimation unit 37 estimates thecause of an abnormality in the camera 6 using the abnormality occurrencetime point of the camera 6 specified by the robot operationdetermination unit 36, the image processing unit 71, or the like, theenvironment information within the data stored in the storage device 31and information on the change with time thereof, the image processinginformation and the information on the change with time thereof, and theimage signal as a group of pieces of input data and displays theestimation result on the display unit 4.

In the abnormality cause estimation unit 37, possible causes ofabnormalities in the camera 6 are defined as and classified into aplurality of (for example, seventeen) abnormality cause items. Theseseventeen abnormality cause items can be grouped into Items 1 to 13belonging to an environmental cause group in which the cause ofabnormalities in the camera 6 originates from an environment and theother causes, that is Items 14 to 17 belonging to an internal causegroup in which the cause of abnormalities in the camera 6 originatesfrom a failure in the camera 6 or a device (a cable, an image processingunit, and the like) connected to the camera 6. Moreover, the thirteenenvironmental items defined to subdivide the environment information isused as Items 1 to 13 belonging to the environmental cause group. Thatis, the abnormality cause items 1 to 13 are the same as environmentalitems 1 to 13 described above, respectively. A failure in the camera 6(more specifically, a failure due to a cloudiness or a dirt on a lens,for example) is classified to Item 14 belonging to the internal causegroup. A failure in the image processing unit 71 is classified to Item15. A failure in the cable connecting the camera 6 and the imageprocessing unit 71 (more specifically, a failure due to disconnection ofa cable or noise associated with the cable) is classified to Item 16. Afailure in the robot 5 (more specifically, a failure due to a decreasein positioning accuracy of the robot 5 or a synchronization errorbetween the robot 5 and the camera 6) is classified to Item 17.

The abnormality cause estimation unit 37 calculates the strength ofcorrelation between an abnormality in the camera 6 estimated to haveoccurred at an abnormality occurrence time point and predeterminedrespective abnormality cause items (more specifically, the percentageindicating the probability that the respective abnormality cause itemsare the causes of abnormalities in the camera 6) using the input datagroup input earlier than the abnormality occurrence time point.

Since the input data group includes information on the environment ofthe camera 6 correlated with the function of the camera 6, the inputdata group and the abnormality in the camera 6 have a correlation. Forexample, when an abnormality occurs in the camera 6 due to excessivevibration of the camera 6 due to a certain reason, there is a strongcorrelation between the occurrence of the abnormality in the camera 6and a change in the information on the vibration of the camera 6belonging to Item 1 of the environment information. Moreover, forexample, when an abnormality occurs in the camera 6 due to a rise in thetemperature of the peripheral device of the camera 6, there is a strongcorrelation between the occurrence of the abnormality in the camera 6and a change in the temperature of the peripheral device belonging toItem 3 of the environment information. Furthermore, for example, when anabnormality occurs in the camera 6 clue to a strong glow of evening sunentering the light receiving surface of the camera 6, there is a strongcorrelation between the occurrence of the abnormality in the camera 6and information on the date and the time belonging to Item 5 of theenvironment information. The abnormality cause estimation unit 37 learnsa correlation among such environment information, environmental changeinformation, the image processing information, and the abnormalities inthe camera 6 using an existing machine learning algorithm to therebycalculate the probability that the respective abnormality cause itemsare the causes of the abnormalities in the camera 6.

A specific example of the machine learning algorithm used in theabnormality cause estimation unit 37 includes a layeredmulti-input-multi-output-type neural network which includes an inputlayer, an intermediate layer, and an output layer, each layer beingformed by coupling a plurality of neurons having predeterminedinput-output characteristics, for example. More specifically, when aninput data group made up of an abnormality occurrence time point of thecamera 6, a plurality of pieces of environment information in apredetermined period between the abnormality occurrence time point and apredetermined previous time point and information on the change withtime thereof, a plurality of pieces of image processing information inthe predetermined period and information on the change with timethereof, and a plurality of image signals in the predetermined period isinput to each neuron that forms the input layer, the abnormality causeestimation unit 37 outputs the probability that the respectiveabnormality cause items are the causes of the abnormality in the camera6.

FIG. 3 is a diagram, illustrating an example of an estimation resultobtained by the abnormality cause estimation unit 37 displayed on amonitor of the display unit 4. As illustrated in FIG. 3, the abnormalitycause estimation unit 37 calculates the probabilities that therespective abnormality cause items are the causes of abnormalities inthe camera 6 using the above-described machine learning algorithm anddisplays the numbers of abnormality cause items and the probabilities onthe display unit 4 in descending order of probabilities. By doing so,since an operator can ascertain the causes of abnormalities indescending order of abnormality cause items displayed on the displayunit 4 (in the example of FIG. 3, in the order of Items 1, 3, and 4), itis possible to specify the cause of the abnormality in the camera 6 morequickly than the conventional method in which a presumed cause ischecked by a round-robin method.

Second Embodiment

Next, a second embodiment of the present invention will be describedwith reference to the drawings. An abnormality cause estimation system1A according to the present embodiment is different from the abnormalitycause estimation system 1 of the first embodiment in that the means forspecifying the abnormality occurrence time point of the camera 6 isdifferent from that of the first embodiment.

FIG. 4 is a functional block diagram illustrating a functionalconfiguration of an arithmetic unit 32A of the abnormality causeestimation system 1A according to the present embodiment. The arithmeticunit 32A includes a robot operation determination unit 36A, anabnormality cause estimation unit 37A, and a camera abnormalitydetection unit 38A and estimates the cause of abnormalities in thecamera 6 using these elements. Hereinafter, the functions of therespective modules 36A to 38A configured to be realized by operations ofthe arithmetic unit 32A will be described sequentially.

The robot operation determination unit 36A determines whether the robot5 can realize an appropriate function using the operation resultinformation among the pieces of data stored in the storage device 31.More specifically, the robot operation determination unit 36 determineswhether a conveying operation executed by the robot 5 is appropriate onthe basis of the image signal transmitted from the camera 6. When it isdetermined that the conveying operation of the robot is not appropriate,the robot operation determination unit 36A transmits information on atime point at which the inappropriate operation was executed to thecamera abnormality detection unit 38A. A specific procedure of the robotoperation determination unit 36A determining whether the conveyingoperation of the robot is appropriate or not is the same as that of therobot operation determination unit 36 of the first, and the detaileddescription thereof will be omitted.

The camera abnormality detection unit 38A detects an abnormality in thecamera 6 using the determination result obtained by the robot operationdetermination unit 36A and the image processing information within thedata stored in the storage device 31. More specifically, when the robotoperation determination unit 36A determines that the conveying operationof the robot is not appropriate, the camera abnormality detection unit38A determines whether the cause of the inappropriate conveyingoperation results from an abnormality in the camera 6 using the imageprocessing information obtained earlier than the execution time point ofthe inappropriate conveying operation and transmits the abnormalityoccurrence time point of the camera 6 to the abnormality causeestimation unit 37A.

As described above, since the image processing information includesinformation used directly for the robot control unit 72 to control therobot 5, there is a correlation between the image processing informationand the abnormality in the conveying operation of the robot 5. Morespecifically, for example, when the contrast of an outline of theworkpiece W included in the image processing information has decreasedexcessively or the image has been brightened excessively, the robotcontrol unit 72 may be unable to realize an appropriate conveyingoperation. The camera abnormality detection unit 38A learns acorrelation between such image processing information and theabnormality in the conveying operation of the robot 5 using an existingmachine learning algorithm to detect an abnormality in the camera 6using the image processing information.

A specific example of the machine learning algorithm used in the cameraabnormality detection unit 38A includes a layeredmulti-input-single-output-type neural network which includes an inputlayer, an intermediate layer, and an output layer, each layer beingformed by coupling a plurality of neurons having predeterminedinput-output characteristics, for example. More specifically, when aninput data group made up of an output (the execution time point of theinappropriate conveying operation) of the robot operation determinationunit 36 and a plurality of pieces of image processing informationobtained in a period between the execution time point of theinappropriate conveying operation and a predetermined previous timepoint is input to each neuron that forms the input layer, the cameraabnormality detection unit 38A outputs the abnormality occurrence timepoint of the camera 6 to the neuron of the output layer.

When the abnormality in the camera 6 is detected by the cameraabnormality detection unit 38A, the abnormality cause estimation unit37A estimates the cause of the abnormality in the camera 6 using theabnormality occurrence time point specified by the camera abnormalitydetection unit 38A, the environment information within the data storedin the storage device 31 and information on the change over timethereof, the operation result information and information on the changeover time thereof, and the image signals as an input data group anddisplays an estimation result on the display unit 4. A specificprocedure of the abnormality cause estimation unit 37A estimating thecause of the abnormality in the camera 6 using the input data group isthe same as that of the abnormality cause estimation unit 37 of thefirst embodiment, and the detailed description thereof will be omitted.

While an embodiment of the present invention has been described, thepresent invention is not limited thereto.

EXPLANATION OF REFERENCE NUMERALS

-   S: Production system.-   1, 1A: Abnormality cause estimation system-   2: Environment information acquisition means-   3: Information processing device-   31: Storage device-   32, 32A: Arithmetic unit-   33: Display unit (Display means)-   36, 36A: Robot operation determination unit (Abnormality occurrence    time point specifying means)-   37, 37A: Abnormality cause estimation unit (Abnormality cause    estimation means)-   38A: Camera abnormality detection unit (Abnormality occurrence time    point specifying means)-   5: Robot (Production device)-   6: Camera (Visual sensor)-   7: Controller-   71: Image processing unit (Image processing means,    Abnormality-occurrence time point specifying means)-   72: Robot control unit

What is claimed is:
 1. A visual sensor abnormality cause estimationsystem for estimating causes of abnormalities in a visual sensor in aproduction system including a production device, the visual sensor thatdetects visual information on the production device or a surroundingthereof, and a controller that controls the production device on thebasis of the visual information obtained by the visual sensor, thevisual sensor abnormality cause estimation system comprising: anenvironment information acquisition means that acquires environmentinformation of the visual sensor; and an abnormality cause estimationmeans that estimates a strength of correlation between an abnormality inthe visual sensor and each of a plurality of predetermined abnormalitycause items using the environment information acquired by theenvironment information acquisition means and displays the estimatedstrength of correlation on a display means for the respectiveabnormality cause items.
 2. The visual sensor abnormality causeestimation system according to claim 1, wherein the abnormality causeestimation means estimates the strength of correlation using theenvironment information in a period between an abnormality occurrencetime point of the camera and a predetermined previous time point.
 3. Thevisual sensor abnormality cause estimation system according to claim 2,further comprising: an abnormality occurrence time point specifyingmeans that specify the abnormality occurrence time point on the basis ofthe visual information.
 4. The visual sensor abnormality causeestimation system according to claim 2, further comprising: an imageprocessing means that performs a predetermined image processingoperation on the visual information; and an abnormality occurrence timepoint specifying means that specifies the abnormality occurrence timepoint using image processing information obtained through the imageprocessing operation and operation result information on a result ofoperations performed by the production device on the basis of the visualinformation.
 5. The visual sensor abnormality cause estimation systemaccording to claim 1, wherein the environment information acquisitionmeans acquires subdivides the environment of the visual sensor into aplurality of environmental items and acquires environment informationbelonging to each of the environmental items, and each of theabnormality cause items is the same as one of the plurality ofenvironmental items.
 6. The visual sensor abnormality cause estimationsystem according to claim 5, wherein the environmental item includesvibration of a specific device installed around the visual sensor, atemperature of the specific device, and a brightness of the surroundingof the visual sensor.
 7. The visual sensor abnormality cause estimationsystem according to claim 1, wherein the abnormality cause estimationmeans estimates the strength of correlation between an abnormality inthe visual sensor and each of the plurality of abnormality cause itemsusing the environment information and information on the change overtime thereof as an input.